The Impact of Cognitive Styles and Educational Computer Environments on Learning Performance

The impact of three treatments Text, CD-ROM, Web site and student Cognitive Styles on learning performance was investigated. Prior to the commencement of the course the students completed the following psychometric tests: the Group Embedded Figures Test (GEFT) Witkin (1971), Kirton’s Adaptive-Innovator Inventory (KAI) Kirton (1976) and Cognitive Styles Analysis (CSA) (Riding 1991). The Group Embedded Figures Test measures the Cognitive Style dimension of Field Dependence-Field Independence. Kirton’s Adaptor-Innovator Inventory locates an individual’s position on the Adaptor-Innovator continuum. The Cognitive Styles Analysis measures an individual’s position on two Cognitive Style dimensions VerbaliserImager (V–I), Wholist-Analytic (W –A) and was intended to subsume many other cognitive styles. The subject matter was an Introductory Course in Artificial Intelligence. Forty-seven final year Information System students participated, randomly assigned within sexes to one of the three treatments. Upon completion of six one-hour sessions they were given a sixty-minute examination to assess overall learning. It was found that only Field Dependence-Field Independence interacted with overall learning performance at a statistically significant level (p < 0.001) irrespective of treatment. When the three treatments were investigated separately the results suggest that Verbalisers performed better than Imagers in the Web treatment, while the Imagers performed better than the Verbalisers in the CD-ROM treatment. The results of the study are discussed in terms of experimental constraints and individual differences together with implications for Web design and continued work. Search Terms GEFT, KAI, CSA, Individual differences, e–learning, Cognitive Styles, learning performance Introduction Cognitive Styles research is a significant predictor of an individual’s educational success. Cognitive Styles provide valuable insights into how individuals apparently learn. Sternberg and Grigorenko (1997 p701) suggest, “cognitive styles seem to have serious implications for educational theory and practice” . In our study, Cognitive Styles and their impact on learner success are investigated in three different educational Computer Assisted Environments namely Text presented via floppy disk, Multimedia CD-ROM and the Web. Computer Assisted Environments provide opportunities for discovery learning and the facilitation of individual differences such as Cognitive Styles. To quote Bruner: “Emphasis on discovery learning has precisely the effect on the learner of leading him to be a constructionist to organize what he is encountering in a manner not only designed to discover regularity and relatedness, but also to avoid the kind of information drift that fails to keep account of the uses to which information might have to be put” (Bruner 1962 p 87). What we wanted to find out was whether the impact of these three environments on learning success could be meaningfully measured and differentiated. Study Design & Rationale Hypermedia plus Hypertext have become important educationally in the last decade or so. The terms hypertext and hypermedia are often used in an interchangeable manner. Hypertext is a non-linear system consisting of textual information nodes and links. Hypermedia encompasses hypertext and facilitates the incorporation of information in a variety of other media formats, video, graphical and pictorial. The WWW (World Wide Web) is an example of a hypermedia system, linking the user potentially to millions of databases globally. Hypermedia is often described as a computer based method of non-sequential reading and writing – a technique with which chunks of nodes of information can be arranged and re-arranged according to an individual’s needs, previous knowledge and curiosity (Borsoek, Higginbotham and Wheat 1997). Hypermedia and Web based learning provide an opportunity for the learner to engage in a personal, internal constructivist learning environment. Being the ultimate interactive learning environment, it presents exciting opportunities for individualized learning in the discovery learning tradition Some eminent educational philosophers and psychologists advocated the discovery learning pedagogical approach e.g. . Dewey; Rousseau, Pestalozzi, Piaget and Bruner. Papert has said “The scandal of education is that every time you teach some thing, you deprive a child of the pleasure and benefit of discovery” (Papert 1996 p68). Jonassen (1989) argues that a well implemented and designed hypermedia system can provide an educational environment that (a) is computer based, (b) exploits multi-nodal approaches to learning and (c) facilitates user decision making opportunity. Central to the concept of hypermedia is the assumption that the mind learns by association (Bush 1945) and that the objective of a well-designed hypermedia system is to allow individuals to create their own associations. We can rank computer learning environments in terms of their historical evolution which shows more and more use of hypertext. In the first case we look at an educational product delivered via floppy disk e.g. an Introductory course on Artificial Intelligence. This may be loaded into main memory and run from there. There may be some use of hyperlinks and hypermedia but this media option is generally limited because of the inherent limitations of floppy disk (one floppy typically holds 1.44 Mbytes). The second media option historically is presenting the material via CD-ROM. A CD-ROM has capacity of the order of 700 Mbytes and as a consequence allows much more scope for the use of graphics, audio and video. The Introductory AI course could be presented via text plus the addition of graphics, sound and video. The third media approach historically is the World Wide Web site. The Web uses hyperlinks and hypermedia much more intensively than the previous two modes. Indeed it is hard to conceive of the Web working without hyperlinks. We are interested in the affect of a user’s cognitive style on performance depends on the medium used. An additional item of interest was how cognitive style affected web navigation strategy. Texst on floppy disk(s) was chosen to represent the closest computer medium to a conventional book with no hyperlinking. This medium is also severely restricted by its memory limitations. An example would be a cookery book or a book on Manet. A CD-ROM can can handle text similarly to floppies but because of its much greater memory can also present graphics, audio and video to enhance the presentation of the textual material. In addition hyperlinking is usually higly used. In our cookery book example graphics, pictures, audio and vissual material can be used to demonstrate the putting together of a recipe with the various ways of handling the individual constituents. The main problem with a CD-ROM is that the material is frozen at a point in time. A web site can be regarded as a superior CD-ROM with much more hyperlinking possible to a very large number of pertinent sites. The web also has the advantage of almost infinite memory and it is also up-to-the minute not just frozen in time as much as the other two media. It is easier to update and be immmediately available. The disadvantage of the Web is access and speed which is improving over time as well as possible contention for the site with other users. The most important aspects of the three media may be summarised as follows: Memory Restriction Hyperlinking Content User sophistication User Contention LearnerCo ntrol Content Restructurability Text 1.4 Mbytes Small if any Core Little No Little Little CD-Rom 700 Mbytes Large Core + AV More No A lot Little Web-site Unlimited Unlimited Core + AV Highly Likely Unlimited Much scope While the course content for each medium remained the same, the presentation was adjusted and extended to take account of the natural advantages of the medium. Study Implementation Forty-seven final year Information Systems students agreed to participate in this study. They were taught an introductory course on Artificial Intelligence designed by one author (AP) over a three-year period. The course material was presented in three computer-assisted formats, Text via Floppy Disk, Multimedia CDROM and a Web based environment. The students attended over a six-week period, totaling seven hours in duration. The students were randomly allocated to one of the three treatments and their attendance was recorded. The students were predominantly male (n = 30). The attendance for all seven sessions was in general good but not 100%. Treatments The subject matter for all three treatments were an Introductory course in AI. While the basic text remained the same the presentations were enhanced to allow for the natural advantages of each medium. All treatments used in the study were run on the same machine type with identical CPU speed and memory in the computer laboratories in Trinity College. The students being IT students were all computer-literate and had no problems using a floppy disk, a CD-ROM or the Internet to access the appropriate treatments. The treatments used are briefly as follows: Treatment 1 – Text via Floppy Disk A set of course notes were presented to the students in a textual, electronic format produced in Microsoft Word on a floppy disk. The subject matter was identical to that presented in treatments two and three. The students loaded the disk that they were given at the commencement of each session and interacted with the text in a passive manner. The speed at which they accessed each page of the text was determined by the speed of the machine. The only linking was between modules. Treatment 2 – Multimedia CD-ROM Essentially the same set of course notes were presented to the students on a CD-ROM which they loaded at the commencement of each session. The multimedia package was designed using Authorware 4 and contained additional text, audio recordings and video clips relative to the floppy disk approach. Hyperlinking existed to a fair extent between abd within modules. Treatment 3 – Web-based Environment An educational web site was developed using Microsoft Front Page 98. At the commencement of each session the students logged on to this site using a password. The core subject matter was identical to that presented in treatments one and two with the addition of numerous links where relevant to other pertinent sites on the WWW. This site was interactive and multimedia in nature containing video, audio and additional textual information. A high degree of Hyperlinking was available within the course module and, of course, to the WWW itself. Learning Performance – Evaluation The students’ knowledge of the material was examined at the end of six hours of instruction. The test comprised two sections A and B. Section A contained thirty multiple-choice questions. Section B of the test consisted of five short objective style questions. Section A was weighted 60%, Section B 40% of the examination total. We were also interested in monitoring the navigational patterns of the students in the Web based environment. A step was considered linear if the user moved from page to page, sequentially. A non-linear step was defined as being a step where the individual left the opened page to explore a nonsequential page. Using Authorware-4 allows for a tracking function for CD-Rom navigation. Unfortunately the designed function would not work in practice. On the WWW (World Wide Web) links that pertained to other Web pages are usually linked in a nonsequential manner. These data were trapped by a specially designed piece of software a cookie. The data was analysed using CGI scripts resulting in a tabulated student navigational log for each of the sessions.The average number of linear and non linear steps for the six sessions was calculated for each student and entered in Minitab .The data was analyzed using simple regression. Due to practical conditions caused by timetabling students for the course, when many students access the same web page at the same time it may overload the server and cause a crash. This also gives rise to problems for the cookie in trying to log the activities of many students.In the measurement session it was necessary for some machines to be booted twice. Extra time was allowed to cover for the loss of time. This illustrates one of the technical difficulties of using a web site for experimental work. Cognitive Styles The construct of Cognitive Style was originally proposed by Allport (1937) referring to an individual’s habitual way of perceiving, remembering, thinking and problem solving. This definition is in pointed contrast to Riding and Rayner (1998 p15) who define Cognitive Style as being, in a much more limited manner, “an individual’s preferred and habitual approach to organizing and representing information “. The cognitive styles studied in this work are: Field dependence-Field Independence (FD-FI), Kirton’s Adaptor-Innovator Inventory (A-I), Verbalisers-Imagers (V-I) and Wholist-Analytic (W-A) which are discussed in more detail next. Field Dependence – Field Independence The concept is probably the most researched of all Cognitive Styles with at least 4,000 publications (Cox and Gall 1981) in the area of Field Dependence-Field Independence. This concept describes the ability of an individual to identify a specific element from within a complex field. Witkin, Goodenough and Cox’s (1977) definition of Field Independence is “the extent to which a person perceives part of a field as discrete from the surrounding field as a whole rather than embedded in the field; the extent to which a person perceives analytically”. Field independent individuals will also tend to re-organize, restructure or represent information to suit their own needs, conceptions or perceptions Jonassen and Grabowski (1993). This difference between these two types of people may be simply illustrated by this analogy: field independent people would be able to pick out the different trees in the forest whereas the field dependent person would just see the forest. Individuals who are Field Dependent experience difficulty in discriminating a particular element from within a complex visual field; they will tend to be more global in their approach. Witkin (1977) when describing the implications of Field Independence– Field Independence referred to the fact that field dependent people tend to have more difficulty with learning material that lacks inherent structure and organization. Studies suggest that individuals who are field independent perform better than those who are field dependent in computer assisted and hypermedia assisted environments. Post (1987) found that there was a positive relationship between Field Independence and achievement in a computer assisted environment. Wey & Waugh (1993) revealed that field independent students performed significantly better than field dependent students in a text only treatment. They suggest that a visual graphical environment may be beneficial to field dependents, thus explaining the lack of significant difference when both groups were compared in a text plus graphic treatment. Weller, Repman and Lan (1993) using a hypermedia-based program called the “Computer Ethics Stack Program” showed that field independent students’ scores were significantly higher than those for field dependents. They also report that field dependent students answered fewer questions and accessed more concept explanations than did field independent students. In a further study (1995) they found that field independent students learned more effectively than field dependent students did and they also revealed that learning patterns varied according to their styles. Wang and Jonassen’s (1993) study show that Field Independent students accessed more screens and spent less time per screen, than did Field Dependent students. They found that Field Independent students tended to cover most of the program by quickly skimming through the screens in the process. Kirton’s Adaptor – Innovator Inventory Kirton (1976) developed the KAI, a self-reporting instrument, which locates an individual along a continuum from Adaptor to Innovator. The KAI was originally designed for adults who were working and had experienced life. Riding and Rayner (1998 p34) report the KAI construct as a Cognitive Style reflecting on an individual’s preferred mode for problem solving. Some of the characteristics of the Adaptor are precision, conformity, reliability and discipline; the characteristics of an Innovator are unreliability, searching for alternative solutions to a problem and taking control of unstructured situations Kirton (1976). The differences between an Adaptor and Innovator are well summed up by Riding and Rayner (1998 p.33): “The adaptor ... generally has a preference for doing things better while the innovator will tend to do things differently”. A recent search of the literature suggests that Kirton’s KAI Cognitive Style has not been applied to any great extent in educational environments or in Computer Assisted Instruction. CSA Riding(1991). The CSA is a computerized psychometric test which measures two independent dimensions of cognitive style the Verbaliser-Imager and the Wholist-Analytic(see section on Psychometric Tests used for a more detailed description). Riding's CSA is an attempt to reduce numerous cognitive styles to two fundamental dimensions Verbaliser-Imager and Wholist-Analytic. Verbaliser–Imager (V-I) Riding and Douglas (1993) reported the finding that Verbalisers performed better than Imagers in a text based environment. Riding’s research has shown that Verbalisers tend to perform better than Imagers in learning environments where the material is in a textual and /or auditory format. Imagers similarly tend to outperform Verbalisers where the presentation mode is graphical and visual. Imagers scored higher than Verbalisers in a similar type environment when the learning material was in text and picture format. Other studies carried out by Riding and Watts (1997); Riding, Buckle and Thompson (1989), Riding and Ashmore (1980) are in broad agreement with this finding also. There is evidence to suggest that Verbalisers recall acoustically complex and unfamiliar text, exhibit greater reading accuracy, comprehension and language ability than do Imagers. (Riding and Calvey 1981, (Riding and Dyer 1980), (Riding and Anstey 1982), (Riding, Rayner and Banner 1999). It is interesting to note that not only do individuals who engage in a learning environment compatible with their Cognitive Style perform better, but they also show an intuitive preference for learning material that best suits their Cognitive Style (Riding and Watts 1997, Riding and Reed 1996). Wholist-Analytic (W-A) Riding’s other orthogonal dimension is the Wholist–Analytic dimension and this also is assessed by the CSA. Where an individual is located on the Wholist-Analytic continuum can often cause an interaction between learning material structure and learning performance. Douglas and Riding (1993) found that Wholists did best when the title of a passage was given before the passage, as opposed to at the end. This interestingly enough had little affect on analytics. Other studies confirm this finding (Riding and Sadler –Smith 1992, Riding & Grimley 1999). Psychometric Tests The psychometric tests used were GEFT, Kirton and CSA and are briefly described as follows: GEFT – Group Embedded Figures Test (Witkin et al 1971). All participants were administered the Group Embedded Figures test. This test is a twenty five-item test, which contains three time sections of 2, 2 and 5 minutes, respectively. The test comes in booklet form and the individual is required to trace one of eight simple figures embedded in a visual background of greater complexity. Kirton – Adaptor-Innovator Inventory (1976) This is a selfreporting instrument. It consists of thirty-two items individually scored on a Likert scale from one to five. The KAI places an individual’s preferred Cognitive Style along a continuum from Adaptor to Innovator. The students also completed this test. Cognitive Style Analysis Riding (1991) The CSA is a computer-based test consisting of three sub-tests: the first, which assesses the Verbaliser-Imager dimension and parts two and three, which assess the Wholist-Analytic dimension. The Cognitive Style Analysis (CSA) measures an individual’s position on its two orthogonal dimensions, Verbaliser and Imager, Wholist and Analytic. The Verbal–Imager dimension is indicative of whether the individual tends to represent information, while thinking, verbally or in mental pictures. On the other hand, the Wholist-Analytic style dimension describes whether an individual is inclined to organize information into wholes or parts. This test was administered to the students. Validity and Reliability of the Instruments Used When Witkins GEFT test is examined, there are conflicting studies concerning the issue of its construct validity( Sternberg 1997), (Johassen and Grabowski 1993). Studies suggest that the GEFT is not particularly satisfactory as a measure of Field Independence-Field Dependence since it is related to spatial ability (Goldstein and Blackman 1978), (MacLeod, Jackson and Palmer 1986). We claim this means that the group labeled Field Dependent may include Field Independent subjects who for one reason other than Cognitive Style (e.g. sight defects) did not score highly on the test. Very acceptable reliability coefficients have been reported for Witkin's GEFT (Witkin 1971) Kirton’s KAI has given an subjected to extensive psychometric validation (Clapp 1993), (Taylor 1994) (Kirton 1987). There is little evidence of published literature in the field which demonstrates the reliability of Riding’s Cognitive Styles Analysis (CSA) at this point in time. Riding and Rayner (1998 p.98) make references to numerous studies, which support the construct validity of those fundamental dimensions as assessed by the CSA. These studies provide the supporting evidence which suggest that these two Cognitive Styles (Wholist–Analyst, Verbaliser-Imager) are independent of intelligence, gender, and personality. Evidence is provided of a physiological relationship with Cognitive Style (Riding Glass et al. 1993), (Riding and Raynor 1998 pp. 100–104). Aim of Study Riding and Rayner (1998 p19) consider their Wholist–Analytic dimension to be a fundamental dimension of many Cognitive Styles: “ it is likely that a number of labels and descriptions of cognitive functioning are related to a single dimension of Cognitive Style. In this instance it may be identified as a wholist-analytic dimension of Cognitive Style”. Cognitive Styles such as Field Dependence–Field Independence, Adaptor–Innovator have been identified as part of this fundamental Wholist–Analytic dimension. The characteristics of both Field Dependents and Wholists are similar, as indeed are those pertaining to Field Independents and Analytics. Riding and Rayner (1998 p33) report that “ A list of characteristics associated with the Adaptor–Innovator dimension ... which suggest that Wholists may be innovators and analytics, adaptors”. The literature indicates that Witkin’s Field Dependence-Field Independence may indeed affect navigational strategy. We thought that this would be most apparent in the Web based treatment We also hypothesised that Verbalisers–Imagers may navigate in different ways (Riding and Glass 1993 ;1997). Thus in this study, our general focus was to investigate whether a variety of Cognitive Styles (WholistAnalytic, Verbaliser–Imager ; Field Dependent-Field Independent and Adaptor –Innovator) affected learner performance in three educational environments (a) text via floppy disk (b) multimedia (CD-ROM) and (c) a website. There was also interest in finding out whether Cognitive Styles and learner performance were independent of treatment . The following questions summarise what we wished to learn: Do cognitive styles affect final score irrespective of treatment ? Do cognitive styles affect final score in the text treatment? Do cognitive styles affect final score in the Multimedia – CD-Rom treatment ? Do cognitive styles affect final score in the Web based Treatment ? Do cognitive styles affect individuals’ navigational strategies in the Web based treatment ? Results The data were analyzed via multiple regression analysis using two packages Data Desk V9 and Minitab V6 Multiple regression analysis of the data allows the quantification of the predictive affect of the independent variables (Cognitive Styles and Gender) on final score. We also used scatterplots as an aid in exploratory data analysis. Scatterplots are highly graphical and can indicate trends in the data. The independent or predictor variable (Cognitive Style) is plotted along the x-axis and the dependent variable (final score) along the y-axis. Sometimes when it was obvious from the scatterplot that a predictor (Cognitive Style) might have an effect on the final score, ordinary linear regression was used. This was suggested when the data were too few to justify a multiple regression analysis; linear regression analysis with the same data but with fewer factors was more effective. In some instances the Cognitive Style proved to be a statistically significant factor when this approach was adopted. Results Section 1 Cognitive Styles Independent of Treatment In this section we are interested in answering the question: “is there a relationship between Cognitive Styles and Final score irrespective of treatment type? “ Table 1 and Figure 1 clearly illustrate that Field Dependence-Field Independence was a significant predictor of final score.